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INFORMATION PROCESSING APPARATUS AND RECORDING MEDIUM 

Incorporation by Reference 

The disclosure of the following priority application is herein incorporated by 
reference: Japanese Patent Application No. 09-159528, filed June 17, 1997. 
5 BACKGROUND OF THE INVENTION 

1. Field of Invention 

The invention relates to an information processing apparatus and recording 
medium. More particularly, the invention relates to an information processing 
apparatus and recording medium for storing and managing a plurality of types of data 
1 0 including image data and audio data. 

2. Description of Related Art 

Imaging apparatus such as electronic cameras record images as digital data. 
These apparatus in recent years have become very popular. Other electronic 
equipment that has become popular in recent years with the advances of 

1 5 semiconductor technology, are electronic notebooks. Electronic notebooks typically 
manage personal information and schedules. In many instances the electronic 
notebooks include touch tablets which provide a mechanism for detecting pressure 
operations (i.e., contact with the touch tablet) by a user. For example, these 
operations may include the recordation of handwriting input by a user into a digital 

20 format 

In addition to the above electronic equipment, digital video cameras which 
record sound and images are also becoming popular. In these apparatus, audio data is 
recorded either in conjunction with the images and/or as separate data before, during 
or after the images are recorded. However, if additional sound is recorded after image 

25 data is already recorded, that additional sound is recorded as separate audio data from 
the audio data that is already recorded. Thus, when the sound accompanying the 
image is played back, it is necessary to select at least two separate audio data for 
playback. This causes, in many instances, non-continuous sound playback. 

SUMMARY OF THE INVENTION 

30 ft is an object of the invention to provide continuous audio data playback even 

when audio data is added to previously recorded audio data. 
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This and other objects are provided by designating a data set associated with 
recorded audio data and recorded image data. Playing back continuous sound is then 
accomplished by successively recording added audio data as data for the data set to 
which the data of the image belongs. 
5 According to an embodiment of the invention, an information processing 

apparatus includes storage means for storing data sets made of a plurality of types of 
data. The plurality of types of data within a data set are correlated to each other. A 
selection means selects a data set from the data sets stored in the storage means. The 
selected data set includes first audio data. An audio collecting means collects sound 

10 and converts it into second audio data. A control means causes the second audio data 
to be stored in the storage means in addition to the first audio data correlated with the 
data set selected by the selection means. 

The invention can be applied to electronic cameras that record sound in 
addition to other information such as, for example, photographic images and line 

15 drawings (memos). The invention also is applicable to devices other than electronic 
cameras. 

According to another embodiment of the invention, an information processing 
apparatus includes audio collecting means for collecting sound and converting the 
collected sound into audio data. For example, a record button, which functions as an 

20 operation means, can be operated to collect the sound and create the audio data. A 
storage means stores the collected audio data. A selection means selects first audio 
data stored in the storage means. A control means causes second audio data collected 
by the audio collecting means to be stored in the storage means in addition to the first 
audio data selected by the selection means as data of a single data set. 

25 The second audio data is stored correlated with the first audio data when the 

first audio data has been selected by the selection means. When the first audio data 
has not been selected by the selection means, the control means causes the second 
audio data to be stored in the storage means without the first audio data. 

A recording medium can be provided that stores a computer-readable control 

30 program that causes the information processing apparatus to function as described 
above. 

These and other aspects and salient features of the invention will be described 
in or apparent from the following detailed description of preferred embodiments. 



BRIEF DESCRIPTION OF THE DRAWINGS 
The invention will be described with reference to the figures in which like 
reference numerals designate like elements and wherein: 

Fig. 1 is a front perspective view of an embodiment of an electronic camera 
5 according to the invention; 

Fig. 2 is a rear perspective view of the Fig. 1 electronic camera with an LCD 
cover open; 

Fig. 3 is similar to Fig. 2, but with the LCD cover closed; 

Fig. 4 shows an internal portion of the electronic camera of Fig. 1; 
1 0 Figs. 5(A)-5(C) are side views of the electronic camera showing different 

positions of an LCD switch and a power source switch; 

Fig. 6 is a block diagram of the internal electrical components of the electronic 
camera of Fig. 1; 

Fig. 7 is a flow chart showing a method of image recording and audio 
15 recording; 

Fig. 8 shows one manner in which data is recorded on a memory card; 
Fig. 9 shows an embodiment of a display of a table of data sets; 
Fig. 1 0 is a flow chart showing a method of adding audio data to a recorded 
data set; 

20 Fig. 1 1 shows another manner in which data is recorded on the memory card; 

Figs. 12(A)-12(C) show one example of the displays of a table of data sets 
when audio data has been added; and 

Figs. 13(A)-13(C) show additional displays of a table of data sets when audio 
data has been added. 

25 DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Fig. 1 shows an embodiment of an electronic camera according to the 
invention. As seen in Fig. 1, the electronic camera comprises surfaces XI, Yl and Z. 
Surface XI faces an object being photographed, whereas surface Yl is a side surface 
and surface Z is a top surface of the electronic camera 1 . The electronic camera 1 

30 further comprises a viewfinder 2 and a shooting lens 3. The viewfinder 2 confirms the 
photographic range of the object being photographed and the shooting lens 3 collects 
light from the object being photographed. A light-emitting unit 4 is also provided for 
emitting light that illuminates the object being photographed. In preferred 
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embodiments, the shooting lens 3, viewfinder 2 and light emitting unit 4 are 
proximate to the top portion of surface XI . 

Surface XI of electronic camera 1 also includes a red-eye reduction lamp 15, a 
photometry device 16 and a colorimetry device 17. The red eye reduction lamp 15 
5 emits light prior to the light emitting unit 4 emitting light in order to reduce a red-eye 
phenomena. The photometry device 16 measures an amount of light from the object 
being photographed and the surrounding area and outputs the results to a photometry 
circuit 51 . The colorimetry device 17 measures a color temperature of the object 
being photographed and the surrounding area and outputs the results to a colorimetry 

10 circuit 52. Both the photometry circuit 51 and the colorimetry circuit 52 are 
controlled by a CPU 39. 

The surface Yl includes a continuous shooting mode switch 13 and a release 
button 10. The continuous shooting mode switch 13 sets a continuous shooting mode 
during photography. The continuous shooting mode switch 13 indicates a type of 

1 5 photography action when a release button 10 is completely and continuously 

depressed. The release button 10 can detect a half depressed state and a completely 
depressed state. The half depressed state is when the release button is depressed 
approximately half a full stroke and the completely depressed state is when the release 
button is depressed a full stroke. 

20 In embodiments, one frame is photographed when the release button 1 0 is 

completely depressed and the shooting mode switch 13 points to a position "S" (i.e., 
single shooting mode). A single frame is shot even when release button 10 is 
continuously pressed. Eight frames are photographed per second when the release 
button 10 is continuously and completely depressed and the continuous shooting 

25 mode switch 13 points to a position "L" (i.e., low speed continuous shooting mode). 

Thirty frames are photographed per second when the release button 10 is continuously 
and completely depressed and the continuous shooting mode switch 13 points to a 
position "H" (i.e., high speed continuous shooting mode). 

Fig. 2 shows a rear surface X2 of the electronic camera. The viewfinder 2 and 

30 a speaker 5 are provided on the rear surface X2. The speaker 5 outputs sound which 
has been recorded in the electronic camera 1 . A liquid crystal display (LCD) 6 and 
operation keys 7 are also provided on the rear surface X2. A touch tablet 6A is 
provided on the LCD 6 and outputs position information corresponding to a position 
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designated through contact with the touch tablet 6A using, for example, a pen type 
instruction mechanism 41 . The touch tablet 6A is preferably made from transparent 
materials such as resin, glass or the like. In preferred embodiments, the user observes 
the images displayed on the LCD 6 through the touch tablet 6 A. 

The operation keys 7 are actuable via the touch tablet 6 A, and include (i) a 
menu key 7A, (ii) an enter key 7B, (iii) a clear key 7C, (iv) a cancel key 7D and (v) a 
scroll key 7E. Other operation keys also can be used with the invention. 

In embodiments, the operation keys 7 detect user contact with the touch tablet 
6A when data is displayed on the underlying LCD 6. The operation keys 7 are used in 
displaying (or otherwise outputting) previously recorded information and in 
performing photography. When actuated, the operation keys 7 output a signal to the 
CPU 39 corresponding to the operation that is being performed. 

Referring particularly to the operation keys 7, when the menu key 7A is 
depressed, a menu screen for selecting various actions is displayed on the LCD 6. For 
example, a display listing the information that is recorded or items used in selecting 
actions for accomplishing various settings may be displayed. The enter key 7B 
reproduces the recorded information (e.g., image data and audio data) which is 
selected by the user. The enter key 7 may also be used for performing a photography 
action. The clear key 7C erases the selected recorded information from a list of 
recorded information. The cancel key 7D cancels the recording of the recorded 
information and the scroll key 7E scrolls up or down the display contents of the 
LCD 6. 

As shown in Fig. 2, an LCD cover 14 covers the LCD 6 and the touch 
tablet 6A. When the LCD cover 14 is moved downward (e.g., an open position), the 
LCD 6 and the touch tablet 6 A are visible. Also, when the LCD cover 14 is in the 
open position, a power source switch 1 1 located on side surface Y2 is turned ON by 
an arm member 14A of the LCD cover 14. 

A microphone 8 for collecting audio data and an earphone jack 9 for 
connecting to an earphone are also provided on upper surface Z. 

Fig. 3 shows the electronic camera 1 when the LCD cover 14 is positioned in 
the closed position over the LCD 6. Fig. 4 shows the interior components of the 
electronic camera 1 . 
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Referring to Fig. 4, an audio recording button 12 for recording audio data is 
provided. The audio recording button 12 may be positioned at approximately the 
same height as the release button 10. However, the audio recording button 12 and the 
release button 10 may be located at different positions so that when either the release 
5 button 1 0 or the audio recording button 12 is depressed, the other button is not 

erroneously depressed. The power source switch 1 1 turns ON and OFF the electronic 
camera 1 . A CCD 20, provided at the rear of the shooting lens 3, photoelectrically 
converts a photographed image into electrical (image) signals. An in-viewfinder 
display device 26 is positioned within the field of the viewflnder 2. The in-viewfinder 
1 0 display device 26 displays the setting states of various functions of the electronic 
camera 1. 

In order to power the electronic camera 1 , four cylindrical batteries 2 1 , for 
example, are provided below the LCD 6. The electric power stored in the batteries 21 
is supplied to the various camera components when the power is ON. A condenser 22 

15 is provided below the LCD 6 and accumulates the electric charge used when the light 
emitting unit 4 emits light. 

A circuit board 23 is provided proximate to the LCD 6, and preferably 
underneath the LCD 6. A plurality of circuits such as a CPU 39 and a DSP 33 (digital 
signal processor) are provided on the circuit board 23. A removable memory card 24 

20 is provided between the circuit board 23 , the LCD 6 and the batteries 2 1 . The 

memory card 24 stores various information that is input into the electronic camera 1. 

An LCD switch 25 is located proximate to the power switch 1 1 and is ON only 
when the LCD switch 25 is depressed. When the LCD switch 25 is ON, an image of 
the object received by the CCD 20 is displayed on the LCD 6 in real time. 

25 Accordingly, the LCD 6 may be used as an electronic viewfinder. 

Figs. 5(A)-5(C) show side views of the electronic camera 1 with the LCD cover 
14 in various positions. Fig. 5A shows the LCD cover 14 in the open position and the 
LCD switch 25 and power switch 1 1 in an ON state. The LCD switch 25 and the power 
switch 11 are in the ON state due to the arm member 14A moving the switches in a 

30 downward position. When the LCD cover 14 is positioned in the closed position over 
the LCD display 6, the power source switch 1 1 and the LCD switch 25 are in the OFF 
state. However, when the LCD cover 14 is in this position, the power source switch 1 1 
may be switched to the ON state independent of the LCD switch 25. For example, as 
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seen in Fig. 5(C), the power source switch 1 1 is ON and the LCD switch 25 remains 
OFF when the LCD cover 14 is positioned over the LCD 6. Because the power source 
switch 1 1 is ON, the electronic circuits are ON and photography actions such as 
photographing an object may be possible, even though the LCD 6 is not displaying 
5 information. 

Fig. 6 shows a block diagram of an embodiment of the internal structure of the 
electronic camera 1. The CCD 20 includes plural light receiving units corresponding to 
a predetermined number of pixels. The CCD 20 photoelectrically converts the light 
incident on the various light receiving units and produces image signals (e.g., electric 

10 signals) corresponding to the predetermined number of pixels. An image processing 
unit (image processor) 31 is controlled by the CPU 39 and amplifies sampled signals 
received from the CCD 20 to a preset level. The sampled signals preferably include 
timing samples of the image signals photoelectrically converted by the CCD 20. The 
CPU 39 also compresses digitized audio information and temporarily stores the 

1 5 digitized information including the compressed audio data in the buffer memory 36 
before storing the data on the memory card 24. 

An analog/digital conversion circuit (A/D converter) 32 digitizes the image 
signals sampled by the image processing unit 3 1 and supplies the digitized image 
signals to the DSP (Digital Signal Processor) 33. The DSP 33 controls a data bus 

20 connected to the buffer memory 36 and the memory card 24. After temporarily storing 
the image data supplied from the A/D conversion circuit 32 in the memory buffer 36, 
the DSP 33 compresses the image data and stores the compressed image data on the 
memory card 24. Accordingly, the buffer memory 36 is used as an input/output buffer 
for the data on the memory card 24. The DSP 33 also stores the image data supplied 

25 from the A/D conversion circuit 32 in a frame memory 3 5 and then displays this data on 
the LCD 6. Similarly, after reading the compressed image data from the memory card 
24, DSP 33 decompresses the image data, stores the decompressed data in the frame 
memory 35 and displays this data on LCD 6. 

The DSP 33 also supplies a CCD horizontal drive pulse to the CCD 20 

30 and controls a CCD drive circuit, (driver) 34 which supplies a CCD vertical drive pulse 
to the CCD 20. During startup of the electronic camera 1, the DSP 33 also causes the 
CCD 20 to adjust an electronic shutter time until an exposure level of the CCD 20 
reaches a proper value. 




8 

An A/D-D/A conversion circuit (converter) 42 converts the analog signal 
corresponding to the audio detected by the microphone 8 into a digital signal. The A/D- 
D/A conversion circuit 42 then outputs the digital signal to the CPU 39. The A/D-D/A 
conversion circuit 42 also converts the digital audio data supplied from the CPU 39 into 
5 an analog audio signal and outputs the analog audio signal to the speaker 5. 

The photometry circuit 5 1 performs a predetermined process on the analog 
signal that is provided from the photometry device 16. In embodiments, the photometry 
circuit 5 1 converts the analog signal into a digital signal and outputs the digital signal to 
the CPU 39. The CPU 39 then controls a stop driving circuit (driver) 53 to change the 
1 0 aperture diameter of the stop 54 in accordance with the photometry results received from 
the photometry device 16. 

The colorimetry circuit 52 performs a predetermined process on the analog 
signal that is received from the colorimetry device 1 7. In embodiments, the colorimetry 
circuit 52 converts the analog signal into a digital signal and outputs the digital signal to 
1 5 the CPU 39. The CPU 39 receives the colorimetry results from the colorimetry device 
17 and adjusts the white balance in accordance with these results. 

A distance measuring circuit (not shown) converts a distance measurement value 
between the object being photographed and the electronic camera, as measured by the 
distance measurement device (also not shown), into a digital signal. The distance 
20 measurement circuit supplies the digital signal to the CPU 39 and the CPU 39 then 

controls a lens driving circuit (driver) 30 in accordance with the signal supplied from the 
distance measurement circuit. This process results in an auto focus action, thus causing 
the shooting lens 3 to move into focus. 

A timer 45 having a built-in timer circuit outputs data corresponding to the 
25 present time (date and time) to the CPU 39. The CPU 39 uses the date and time data 

supplied from the timer 45 and records the information on the memory card 24 as image 
data and audio data management information. 

The stop driving circuit 53 sets the aperture diameter of the stop 54 to a 
predetennined value. The stop 54 is preferably positioned between the shooting lens 3 
30 and the CCD 20 and changes the aperture of light incident on the CCD 20 from the 
shooting lens 3. 

An interface (I/F) 48 is provided so that data can be transmitted between the 
electronic camera and other apparatus such as, for example, a personal computer. 
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A strobe driving circuit (driver) 37 and a red-eye reduction (RER) lamp driving 
circuit (driver) 38 are also provided. The CPU 39 controls the red-eye reduction lamp 
1 5 and light emitting unit 4 to emit a suitable amount of light. However, in preferred 
embodiments, the light ermtting unit 4 does not emit light when the LCD cover 14 is in 
5 the open position so that the photography action may be performed as displayed in the 
electronic viewfinder. 

CPU 39 also controls an in-viewfinder display circuit (driver) 40 to display 
various information on the in-viewfinder display device 26. 

The CPU 39 receives position information from the touch tablet 6A and then 
1 0 processes the information. Specifically, when an area of the touch tablet 6A 
overlapping the LCD 6 is selected, the CPU 39 successively references the 
information of various data stored on the memory card 24 by reading the X-Y 
coordinates of the position pressed on the touch tablet 6A. By way of example, when 
the enter key 7B is pressed, the CPU 39 compresses the line drawing data in the buffer 
1 5 memory 3 6 prior to recording the data on the line drawing data recording area of the 
memory card 24. The line drawing data is also recorded along with the input date and 
time as header information to the line drawing data. In embodiments, the line drawing 
does not have to be compressed when the amount of line drawing information is 
small. 

20 The user also can input line drawing data (memo data) by contacting the touch 

tablet 6A with pen 41. The X-Y coordinate(s) where the touch tablet 6 A is touched is 
used by the CPU 39 to display a corresponding point (or points - such as a solid or 
broken line) on the LCD 6. Since the touch tablet 6A is transparent and overlies the 
LCD 6, it appears as if the user is writing on the LCD 6. 

25 The line drawing data input into the touch tablet 6A contains a large amount of 

components with a high spatial frequency. Because of the large amount of 
components with a high spatial frequency, it is preferable to use a run length method 
to compress the data. The run length method compresses the line drawing and 
reduces information loss when decompressing the compressed line drawing data. This 

30 is accomplished by scanning the line drawing screen in a horizontal direction and 
encoding each continuous string of data (i.e., points) for each color (such as black, 
white, red, blue and the like) and each continuous string of no data (portions with no 
pen input). The run length method is a known method used in facsimile machines. 
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The JPEG (Joint Photographic Experts Group) method can be used to 
compress the shooting image data. Alternatively, other forms of compression (or no 
compression) can be used. 

As will become clearer from the following description, various types of data 
5 can be input and stored by the electronic camera. For example, shooting image data, 
line drawing (memo) data and sound data can be input and stored. Each piece of data 
is stored with header information. Different pieces of data can be associated 
(correlated) with each other by correlating the header information of each piece of 
data. Correlation can be achieved, for example, by saving the same time (date and 

1 0 time) data as the header information (this is possible even-if the various pieces of data 
are not actually input at the same time). Pieces of data that are correlated to each 
other are considered to be members of the same data set. For example, a photographic 
image, a line drawing and a first audio data having correlated (e.g., the same) header 
information are part of a data set. When that data set is reproduced, it is possible to 

15 display the photographic image and the line drawing simultaneously (superposed) on 
LCD 6 while the sound corresponding to the audio data is output by the speaker 5. 

According to the invention, any data set may be selected from among plural 
data sets composed of a plurality of types of correlated data. Assume that the selected 
data set includes first audio data. A second audio data corresponding to collected 

20 audio data may be stored in addition to the first audio data of the selected data set so 
that the sound corresponding to the first and second audio data may be continuously 
reproduced. Accordingly, it is possible to continuously reproduce the sound 
corresponding to the audio data belonging to that data set by designating the data set. 
Fig. 7 is a flow chart showing a method of image recording and sound recording. 

25 In step S 1 the CPU 39 makes a determination as to whether the release button 1 0 has 
been completely depressed. If the release button 10 is not completely depressed, a 
determination is made in step S2 as to whether the audio recording button 12 is 
depressed. If step S2 is negative, the process returns to step SI . Thus, the CPU 39 waits 
until either the release button 1 0 or the audio recording button 12 is depressed. 

30 When the release button 1 0 is completely depressed, the CPU 39 controls the 

recording of the image in step S3. Specifically, the image processing circuit 3 1 samples 
signals corresponding to the image of the object The sampled signals are converted 
into digital image data by the A/D conversion circuit 32 and output to the DSP 33. The 
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DSP 33 temporarily stores the image data in the buffer memory 36, and compresses the 
data, for example, in accordance with the JPEG format. 

In step S4 the CPU 39 determines the number N of data sets recorded on the 
memory card 24. In step S5 the CPU 39 records the image data compressed by the DSP 
5 33 on the memory card 24 as the image data of data set N + 1 . The image data and 

management information such as the number of the data set and the photography date 
and the photography time are recorded on the memory card 24 as header information. 

In step S6 the CPU 39 sets to 1 the value of the counter i used to count the audio 
data corresponding to the predetermined data set In step S7 the CPU resets a time 
10 counter. After the image data is recorded, a determination is made in step S 8 as to 
whether the audio recording button 12 is depressed. 

If the audio recording button 12 is not depressed, the value of the time counter 
is referenced in step S9 and a detennination is made as to whether a preset time 
interval has elapsed from when the image data was recorded. If the preset time 
1 5 interval has elapsed from when the image data was recorded, the process returns to 

step SI and the CPU 39 waits until either the release button 10 or the audio recording 
button 12 is depressed. If the preset time interval has not elapsed from when the 
image data was recorded, the process returns to step S7. 

If the audio recording button 12 is depressed at step S8, the process moves to 
20 step S10 where an audio recording process is performed. In the audio recording 
process, the audio signal collected by the microphone 8 while the audio recording 
button 12 is depressed is converted into digital audio data by the A/D-D/A conversion 
circuit 42. The digital audio data is then output to the CPU 39. 

In step SI 1 the CPU 39 compresses the audio data before recording the data on 
25 the memory card 24 as audio data i of the data set having the same number (N+l ) as 
the image data recorded in step S5. The audio data and management information 
including the number of the data set and the number of the audio data is recorded as 
header information. 

After audio data number i has been recorded on the memory card 24, the CPU 
30 39 increments the value of the counter i by 1 in step S12. The process returns to step 
S7 and the CPU 39 waits for either the preset time interval to elapse or the audio 
recording button 12 to be depressed. 
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Revisiting step S2, when it is determined in step S2 that the audio recording 
button is depressed, the process moves to step SI 3 and the audio recording process is 
performed. This is the same process as described in step S10. 

In step S 14 the CPU 39 finds the number of data sets recorded on the memory 
5 card 24. In step SI 5 the CPU records the audio data on the memory card 24 as the 
audio data of data set number (N + 1). That is, a new data set is provided. It is to be 
noted that the audio data and management information including the number of the 
data set and the recording date and the recording time are recorded on the memory 
card 24 as header information. In step S 16 the CPU 39 sets the value of the counter i 

10 used to count the audio data to 2 because audio data was recorded in step SI 5. The 
process then moves to step S7 and the same process from step S7 to step SI 2, as 
previously described, is performed. 

By using the above process, audio data input before a preset time has elapsed 
from when (i) the image data was recorded, (ii) the audio data accompanying the 

15 image data was recorded, or (iii) independent audio data was recorded, is recorded on 
the memory card 24 as audio data of the data set to which the previous data belongs. 
That is, audio data input before a preset time has elapsed from when image data or 
audio data was input is stored as a continuous audio sequence. 

Fig. 8 shows an embodiment of various types of data being recorded on the 

20 memory card 24 assurning no data sets have been previously recorded (N=0). Image 
data is input by depressing the release button 10. Input of sound is followed by 
depressing the audio recording button 12 before a preset time interval (e.g., 3 seconds) 
has elapsed. The image data and the audio data corresponding to the input image and 
sound are recorded on the memory card 24 as the image data and audio data of data 

25 set 1 . The management information of the image data and the audio data are also 
recorded in data set number 1 . 

After a preset time interval has elapsed from when the audio data is recorded, 
image data is again input by depressing the release button 10. The image data is 
recorded as the image data of data set 2. After a preset period of time has elapsed 

30 from when the image data of data set number 2 is recorded, audio data is input by 
depressing the audio recording button 12. The input audio data is recorded as the 
audio data of data set 3. After a preset time interval has elapsed from when the audio 
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data of data set 3 is recorded, sound is again input by depressing the audio recording 
button 12 and is recorded as the audio data of data set number 4. 

Fig. 9 shows the LCD 6 having a data set number, recording date and time, 
thumbnail image, audio mark and audio time for each collected data set. In this 
5 example, data set number 1 includes image data corresponding to an image recorded 
at 8:30 and audio data corresponding to audio that is three seconds long. Data set 
number 2 includes image data corresponding to an image recorded at 8:32. Data set 
number 3 includes audio data corresponding to sound that is five seconds long and 
recorded at 9:25. Data set number 4 includes audio data corresponding to sound that 
1 0 is two seconds long and recorded at 9:27. When line drawing data is included in a 
data set, a mark unrepresented in Fig. 9, is displayed as one part of the data for that 
data set. 

Fig. 10 is a flow chart of a process when audio data is added to a recorded data 
set. In step S21 a user selects a data set (e.g., by touching the data set number with 

1 5 pen 41) and the CPU 39 receives the signal corresponding to the pressed operation 
and reverses the display colors of the display of that data set. In step S22 when the 
audio recording button is pressed by the user, the A/D-D/A conversion circuit 42 
receives the audio data corresponding to the sound collected by the microphone 8. In 
step S23 the CPU 39 makes a determination as to whether audio data is included in 

20 the data set selected by the user. When the determination is that audio data is not 
included in that data set, the number of the audio data is set to 1 in step S24. 

When it is determined that the audio data is included in the data set, the 
process moves to step S25 and the CPU 39 references the management information of 
each audio data item on the memory card 24 to find the number M of audio data items 

25 included in that data set. In step S26 the process sets the number (M+l). InstepS27 
the process creates management information such as (i) the number of the audio data, 
(ii) the number of the selected data and (iii) the date and the time data supplied from 
the timer 45. This is performed after the number of the audio data corresponding to 
the recorded sound is set. In step S28 the process records the audio data on the 

30 memory card 24 along with the management information. 

By utilizing the above process, audio data may be added to the selected data 
set. That is, audio data having as one part of the management information the number 
of the selected data set and the number of the audio data that is one number larger than 
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the number of audio data items previously belonging to that data set may be recorded 
on the memory card 24. 

By way of example, when the audio recording button 12 is depressed, sound is 
collected by the microphone 8. The audio data corresponding to the collected sound 
5 is supplied to the CPU 39. The audio data is then compressed and recorded on the 
memory card 24 as audio data 2 of data set 1 (Fig. 1 1). When the recorded time is 5 
seconds, the value of the audio time and the display of the data set display on the LCD 
6 is changed from 3 to 8 (Figs. 12(A) and 12(B)). That is, the sum of the length of the 
audio items corresponding to all of the audio data belonging to data set 1 is displayed 
10 as the value for that audio time. 

When the data set number 1 is again selected, and sound that is two seconds 
long is recorded, and the audio data corresponding to that sound is recorded on the 
memory card 24 as audio data 3 of data set 1 (Fig. 11). The value 1 0 is displayed as 
the audio time of data set 1 (Fig. 12(C)). Note that the display of the recording date 
1 5 and time remains at 8:30 even if audio data is added (Fig. 12(C)). However, it is also 
possible to update this time to the time when the new audio data is added. 

When the above described data number 2 is added to data set number 1 at 
9:25, the display of the recording date and time for data set number 1 can change from 
that shown in Fig. 13(A) to 9:25 (Fig. 13(B)). When the audio data number 3 is added 
20 to data set number 1 at 9:27, the display of the recorded data and the time for data set 
1 changes to 9:27 (Fig. 13C)). This provides continuous sound playback 
corresponding to an associated image, regardless of when the sound is recorded. 

Although the JPEG and run length encoding compression techniques were 
described, other compression techniques (or no compression at all) can be used with 
25 the invention. 

Although a touch tablet with input pen were described as structures through 
which selections and commands can be input, the invention is not limited to such 
structure. For example, the touch tablet can be actuable by the user's finger. 
Additionally, selections and commands can be input without using a touch tablet. For 
30 example, a cursor can be moved (e.g., via a mouse) and selections or commands can 
be made by clicking. 

The invention is not limited to implementation by a programmed general 
purpose computer as shown in the preferred embodiment. For example, the invention 
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can be implemented using one or more special purpose integrated circuits) (e.g., 
ASIC). It will be appreciated by those skilled in the art that the invention can also be 
implemented using one or more dedicated or programmable integrated or other 
electronic circuits or devices (e.g., hardwired electronic or logic circuits such as 
discrete element circuits, or programmable logic devices such as PLDs, PLAs, PALs 
or the like). In general, any device or assembly of devices on which a finite state 
machine capable of implementing the flow charts shown in Figs. 7 and 10 can be 
used. 

While this invention has been described in conjunction with specific 
embodiments thereof, it is evident that many alternatives, modifications and variations 
will be apparent to those skilled in the art. Accordingly, the preferred embodiments of 
the invention set forth herein are intended to be illustrative, not limiting. Various 
changes may be made without departing from the spirit and scope of the invention as 
defined in the following claims. 



